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DETAILED ACTION 



1 . This action is responsive to communications of amendment received 1 1/10/2004. 

2. The disposition of the claims is as follows: claims 1-15 are pending in the application. 
Claims 1 and 13 are independent claims. Claims 16-25 drawn to non-elected invention have 
been cancelled. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bushey, 
(PGPUB-DOCUMENT-NUMBER: 20030001851), and further in view of Cunningham et al., 
(US 5,041,992 A). 

A. Bushey discloses claim 1, "An image analysis and conversion method comprising: 
receiving a bitmapped image; and determining perceptually salient structures in the bitmapped 
image; each perceptually salient structure correlating to a distinct area of the bitmapped image; 
converting the determined perceptually salient structures into structured object representations of 
the bitmapped image, the structured object representations being symbolic representations of the 
perceptually salient structures which are editable by a structured text/graphics editor [para. 0005 
at ' Recognition software exists today which will transform a bit map or raster graphics digital 
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image into an object-oriented graphic representation of the digital image. These algorithm vary 

in capability and_the_graphical_representation 

capability of the algorithm used in the conversion. For instance, if this algorithm is used on the 
digital image of a person, the quality of the appearance of the graphic image of the person is a 
direct function of the quality of the algorithms which convert the raster digital image of the 
person into an object-oriented graphical representation of the person.'; and 

para. 0010, at 'The need for effectively and efficiently processing both object-oriented 
and bit map graphics and images is achieved by a system for and method of combining both 
object-oriented (or "vector") graphics and raster scan (or " bit-mapped" or "bit map" image ) 
pipelines so that image data can be processed by selected stages of either and/or both pipelines. 
That is, an object-oriented graphics pipeline may include several processing stages configured to 
sequentially process graphics data, the graphics data may include data to generate one or more 
graphic primitives, such as points, lines, polygons, alphanumeric characters, special symbols and 
fonts, etc. Examples of possible graphic processing stages in the object-oriented graphic pipeline 
may include some combination or subcombination of scan conversion, clipping, windowing to 
viewport, projection, sorting and/or may include other or substitute functionalities and others. 
Similarly, a bit-mapped or raster scan pipeline may include several processing stages configured 
To sequentially process bit-mapped data representing an image as a plurality of pixel luminance 
values and/or other pixel characteristics. These bit-mapped or raster scan pipeline processing 
stages may include some combination or subcombination of demosaicing, color correction/white 
balancing, gamut mapping, tone correction, flare correction, color transformation, scaling and/or 
may include other or substitute functionalities and others. Advantageously, the processor 
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provides for selective sequencing through each pipeline and for "cross-over" of data between 
stages of eac h pipeline including any_nec,essary,dataxonversti^ — 
conversion of bit-mapped images to form object-oriented graphic data, and similar conversion of 
object-oriented graphic format data into a bit map data format.); and 

para. 0013 at 'According to another feature of the invention, the graphics pipeline is 
configured to receive graphics data including graphics identification and location data and the 
bit-mapped image pipeline is configured to receive a raster scanned image data representing 
pixel luminance information. A data format converter may be included and configured to 
convert between an object-oriented graphics data format and a bit map image data format, i.e., 
unidirectionally from one, or bidirectionally from either format to the other.'; and 

para. 0014 at 'According to another feature of the invention, an image recognition stage 
may be configured to identify and encode object-oriented graphic images within the bit-mapped 
image . For example, the recognition stage may recognize primitive elements and shapes 
including polygons, polyhedrons, lines, planes, curves, arcs, points, vectors, characters, symbols, 
and other primitives and composite graphic structures within a bit map or raster based 
representation and convert the same into an appropriate object-oriented graphic data 
representation. The conversion may include deletion of the converted portions from the original 
bit map image so that a remaining portion of the latter can be appropriately processed while the 
converted objects are subject to graphics pipeline processing. A common instruction decoder 
may be included, operable to control the interconnection (e.g., a switching matrix) to route at 
least one of the graphic objects and the bit-mapped image object between both the graphics and 
bit-mapped image pipelines. 9 ; and para. 0018 and 0033]" supra [as detailed]. 
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Wherein 'primitive elements and shapes including polygons, polyhedrons, lines, planes, 

cwyes 5 jircs,_p^ 

structures within a bit map or raster based representation' correspond to "perceptually salient 
structures in the bitmapped image"; 'Recognition 5 implies "receiving" having occurred and 
'transform 5 corresponds to "converting". 

Likewise Polimeni et al., (U.S. Patent 5,874,966 A), discloses similar ideas in col. 7, Ins. 40-54 
that [arbitrary, imported, color bitmap image is set as the graphical scene displayed as the 
graphical user interface for the computer. The process proceeds to step 180 where major objects 
in the scene are automatically identified by the computer. A major object is an object shown in 
the selected color bitmap image having a plurality of pixels of a selected characteristic. In 
accordance with one aspect of the present invention, that selected characteristic is the color 
displayed by pixels in the bitmap image . Therefore, the present invention automatically 
identifies major objects in the scene presented by the graphical user interface by identifying 
objects having groups of pixels that are substantially the same color.] 

However, Bushey does not appear to disclose "which are editable by a structured 
text/graphics editor", but Cunningham et al., (US 5,041,992 A), do in col. 6, Ins. 23-52 at [Bit 
maps may be generated directly with bit map editors that display a large pixel representation of 
the bit map upon a display screen and allow the programmer to black in or clear individual 
pixels. Use of such a bit map editor is obviously tedious. Another form of graphics editor is 
known as a object graphics editor . Object graphics editors respond to commands to draw and 
erase lines and circles and other shapes and to fill and clear specified areas. The graphics editor 
builds a list of commands for describing an image. Before the object graphics generated image 
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can be displayed on a bit mapped display, it must be converted to a bit map. Object graphics 
editors often include such a utility. _An, object graphics-editor kno\\^-as-Sketch— is available as-a- 
drawing program for the Interlisp-D environment. It enables the interactive construction of 
figures from a combination of text and graphics. A sketch created by the "Sketch" program 
consists of elements such as text, lines and curves, boxes, circles and ellipses. Each element has 
one or more positions that determine the location and shape and a set of properties that determine 
how it looks. Sketch is an interactive graphics editor in that locations upon the display may be 
designated by use of a mouse cursor and mouse events. Object graphics editors are known in the 
art. "Sketch" is described in A Users Guide To Sketch-The Interlisp Drawing System, Xerox 
Corporation, 1985. "McDraw" is an object graphics editor widely known to Apple Macintosh 
users.] 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply recognition software disclosed by Bushey in combination with 
object graphics editor disclosed by Cunningham et al., and motivated to combine the teachings 
because use of an object graphics editor would not be tedious as opposed to a bit graphics editor, 
as revealed by Cunningham et al., in col. 6, lines 23-28. 

B. Per independent claim 13, this is directed to a system for performing the method of 
independent claim"!, and therefore is rejected to independent claim 1. 

5. Claims 2, 3 and 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bushey, 
(PGPUB-DOCUMENT-NUMBER: 20030001851), further in view of Cunningham et al., (US 
5,041,992 A), as applied to claims 1 and 13 above, and further in view of Withgott et al., (US 
Patent 5,491,160), hereinafter Withgott. 
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A. Bushey and Cunningham disclose claim 2, "The method according to claim 1 wherein the 

j^cxivmg^pjnc^ 

supra for claim 1 . 

However Bushey and Cunningham do not appear to disclose, "wherein the receiving step 
includes, scanning a non-electronic generated image into the bitmapped image", but Withgott 
does in col. 6, In. 64 - col. 7, In. 23 at [The electronic document image to be processed is created 
in any conventional manner, for example by a conventional scanning means such as those 
incorporated within a document copier or facsimile machine, a Braille reading machine, or by an 
electronic beam scanner or the like. Such scanning means are well known in the art, and thus are 
not described in detail herein. An output derived from the scanning is digitized to produce 
undecoded bit mapped image data representing the document image for each page of the 
document, which data is stored, for example, in a memory 15 of a special or general purpose 
digital computer data processing system 13.] 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply recognition software disclosed by Bushey in combination with 
object graphics editor disclosed by Cunningham et al., coupled with a copy scanner disclosed by 
Withgott, and motivated to combine the teachings because the output derived from the scanning 
isHigitized to produce undecoded bit mapped image data representing the document image, as 
revealed by Withgott et al., in col. 7, lines 7-17, and because use of an object graphics editor 
would not be tedious as opposed to a bit graphics editor, as revealed by Cunningham et al., in 
col. 6, lines 23-28. 
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B. Bushey and Cunningham disclose claim 3, "The method according to claim 1 wherein the 
_bita^ppedumgej^e^ 

image" supra for claims 1 and 2. However Bushey and Cunningham do not appear to disclose, 
"wherein the bitmapped image received in the receiving step is an originally generated electronic 
bitmapped image", but Withgott does in col. 6, In. 64 - col. 7, In. 1 at 'With reference first to 
FIG. 2, the method is performed on an electronic image of an original document 5, which may 
include lines of text 7, titles, drawings, figures 8, or the like, contained in one or more sheets or 
pages of paper 10 or other tangible form' wherein 'the method is performed on an electronic 
image of an original document' corresponds to "an originally generated electronic bitmapped 
image". 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply recognition software disclosed by Bushey in combination with 
object graphics editor disclosed by Cunningham et al., coupled with a copy scanner is performed 
on an electronic image of an original document disclosed by Withgott, and motivated to combine 
the teachings because the output derived from the scanning is digitized to produce undecoded bit 
mapped image data representing the document image, as revealed by Withgott et al., in col. 7, 
lines 7-17, and because use of an obj ect graphics editor would not be tedious as opposed to a bit 
graphics editor, as revealed by Cunningham et al., in col. 6, lines 23-28. 

C. Bushey and Cunningham disclose claim 4, "The method according to claim 1 wherein the 
converting step includes, altering the bitmapped image into multiple alternative interpretations" 
supra for claim 1. However Bushey and Cunningham do not appear to disclose, "wherein the 
converting step includes, altering the bitmapped image into multiple alternative interpretations", 
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but Withgott does in col. 22, Ins. 15-27 at 'The determined image characteristic(s), e.g., the 

deri ved ima g^n^shapejepre sentations o f each selected.image-unitare compared, as-noted 

above (step 841), with the determined image characteristic(s)/derived image unit shape 
representations of the other selected image units for the purpose of identifying equivalence 
classes of image units (step 850), such that each equivalence class contains most or all of the 
instances of a given word in the document. The equivalence classes are thus formed by 
clustering the image units in the document based on the similarity of image unit classifiers, 
without actually decoding the contents of the image units, such as by conversion of the word 
images to character codes or other higher-level interpretation.' and in col. 8, Ins. 22-32 at 'A 
morphological operation refers to an operation on a pixelmap image (a source image), that uses a 
local rule at each pixel to create another pixelmap image, the destination image. This rule 
depends both on the type of the desired operation to perform as well as on the chosen structuring 
element.' and furthermore in para. (25, 67, 93 and 1 16). Wherein 'equivalence classes', 
'morphological operation' and 'time warping' implementations, each correspond to "altering the 
bitmapped image into multiple alternative interpretations". 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply recognition software disclosed by Bushey in combination with 
object graphics editor disclosed by Cimningharnet al, coupled with 'equivalence classes', 
'morphological operation' and 'time warping' disclosed by Withgott, and motivated to combine 
the teachings because for automatically generating ancillary document images reflective of the 
contents of an entire primary document image, as revealed by Withgott et al., in col. 3, lines 13- 
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17, and because use of an object graphics editor would not be tedious as opposed to a bit 

graphics_editor,_as_re^^ 

6. Claims 5, 9, 10 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bushey, (PGPUB-DOCUMENT-NUMBER: 20030001851), further in view of Cunningham et 
al., (US 5,041,992 A), as applied to claims 1 and 13 above, further in view of Withgott et al., 
(US Patent 5,491,160), hereinafter Withgott, and further in view of Microsoft Paint, hereinafter 
Paint. 

A. Bushey, Cunningham and Withgott disclose claim 5, "The method according to claim 4 
wherein the altering of the bitmapped image into multiple alternative interpretations includes, 
altering the bitmapped image into informal structured object representations that are editable by 
the structured text/graphics editor, and altering the bitmapped image into formal structured 
object representations that are editable by the structured text/graphics editor" supra for claim 4. 

However, Bushey, Cunningham and Withgott do not appear to disclose "wherein the 
altering of the bitmapped image into multiple alternative interpretations includes, altering the 
bitmapped image into informal structured object representations that are editable by the 
structured text/graphics editor, and altering the bitmapped image into formal structured object 
representations that are editable by the structured text/graphics editor 55 . But Paint does as 
exemplified below in Figs 1-3. Using the 'paste from 5 on the 'Edit 5 drop down menu one can 
import both informal 'cursive h 5 and formal 'textual ON 5 into Paint. Then using dotted rectangle 
to select the informal 'cursive h' and then using the Flip/Rotate on the 'Image 5 drop down menu 
to flip/rotate in the horizontal and vertical directions to create the alternative representation of a 
'cursive y\ Then using dotted rectangle to select the formal 'textual ON 5 and then using the 
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Flip/Rotate on the 'Image' drop down menu to flip/rotate in the horizontal and vertical directions 

to cr eate the alternativerepresentation-of.a ItextuaLNO' 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply recognition software disclosed by Bushey in combination with 
object graphics editor disclosed by Cunningham et al., coupled with 'equivalence classes', 
'morphological operation' and 'time warping' disclosed by Withgott, and in concert with 'paste 
from' and flip/rotate' disclosed by Microsoft Paint, and motivated to combine the teachings 
because for automatically generating ancillary document images reflective of the contents of an 
entire primary document image, as revealed by Withgott et al., in col. 3, lines 13-17, and because 
use of an object graphics editor would not be tedious as opposed to a bit graphics editor, as 
revealed by Cunningham et al., in col. 6, lines 23-28, and since both informal and formal 
structured object representations are easily edited by 'paste from' and flip/rotate' disclosed by 
Microsoft Paint in Figure 3. 
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Figure 2 



a Paint Help 



jaife If jgfas 



To flip or rotate a picture or object 

1 Click IISII or Bi2 in the tool box. and then drag a box around the object or area you want to flip or rotate. 

■ Click [1210 below the tool box to flip or rotate opaayejy. 

■ Click IffifS below the tool box to flip or rotate transparently. 

2 On the Image menu, click Flip/Rotate. 

3 Click the option you want. - 

Notes and Tips --"^ ■ ■ — •- - - - ----- - — ' • ' ™ 

■ Use to select a free-form shape. Use ESii to select a rectangular shape. 

■ To undo a change/click Edit and then click Undo. You can undo up to three changes. 

Ml Related Topics V . ,'.y ^- . 



Figure 3 



About Paint 




B. Per dependent claim 14, this is directed to a system for performing the method of 
dependent claim 5, and therefore is rejected to dependent claim 5. 
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C. Bushey, Cunningham, Withgott and Microsoft Paint disclose claim 9, "The method 
^ortogjojjlamjjyh^ — 
representations includes generating multiple structured object representations of the bitmapped 
image, the multiple structured object representations, representing at least a first image 
representation having formal structured object representations, and a second image 
representation containing informal structured object representations" supra for claim 5. Wherein 

'h' and 'y' correspond to informal and 'ON' and 'NO' correspond to formal. 

D. Bushey, Cunningham, Withgott and Microsoft Paint disclose claim 10, "The method 
according to claim 1 wherein the editing by the structured text/graphics editor permits movement 
of structured object representations by at least one of, individual objects, a sub-group of all the 
structured objects, or as an overall group of the structured object representations" supra for claim 
5. Wherein 'y' and 'NO' correspond to "movement as an overall group of the structured object 
representations". 

7. Claims 6 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bushey, 
(PGPUB-DOCUMENT-NUMBER: 20030001851), and further in view of Cunningham et al., 
(US 5,041,992 A) as applied to claim 1 above, and further in view of Ohmori et al., (US Patent 
6,678,397 Bl), hereinafter Ohmori. 

A. Bushey and Cunningham disclose claim 6, "The method according to claim 1 wherein the 
step of converting the bitmapped image into structured object representations of the bitmapped 
image includes configuring the structured object representations to represent an electronic slide 
of the structured text/graphics editor" supra for claim 1. However, Bushey and Cunningham do 
not appear to disclose "to represent an electronic slide", but Ohmori does in col. 8, Ins. 35-40. 
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Therefore it would have been obvious to one of ordinary skill in the art at the time the 
inve ntion was ma de to apply editing scanned.documentS-disclosed by-Bushey-and Cunningham — 
in combination with representing an electronic slide of the structured text/graphics editor as 
disclosed by Ohmori, and motivated to combine the teachings because it would display as 
revealed by Ohmori in col 8, lines 1-8. 

B. Bushey and Cunningham disclose claim 7, "The method according to claim 1, wherein 
the converting step includes forming of an Alternative graph" supra for claim 1 . However, 
Bushey and Cunningham do not appear to disclose "forming of an Alternative graph", but 
Ohmori does in col. 12, Ins. 1-12. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply editing scanned documents disclosed by Bushey and Cunningham 
in combination with forming of an Alternative graph as disclosed by Ohmori, and motivated to 
combine the teachings because it would show how converting progresses as revealed by Ohmori 
in col. 12, lines 8-12. 

8. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bushey, 
(PGPUB-DOCUMENT-NUMBER: 20030001851), further in view of Cunningham et al, (US 
5,041,992 A), and further in view of Smith et al., (PGPUB-DOCUMENT-NUMBER: 
20040257367), hereinafter Smith. " 

A. Bushey and Cunningham disclose claim 11, "The method according to claim 1 wherein 
the bitmapped image is converted into the structured objects representations of the bitmapped 
image through the use of an Alternative Graph" supra for claim 1. However Bushey and 
Cunningham do not appear to disclose "wherein the bitmapped image is converted into the 
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structured objects representations of the bitmapped image through the use of an Alternative 

Gmph",Jut Simth_doesjn ^ 

simplified representation of one possible graph structure usable to implement an embodiment of 
the invention. One skilled in the art will appreciate that alternative graph structures may 
equivalently be used within the scope of this embodiment of the invention. In the illustrated 
structure, each node 300 represents an operation such as "blur," "rotate," and so on. Each 
operation may have parameters associated with it, such as "amount of blur," "angle of rotation," 
etc. Leaf nodes 302 and 304 contain or generate image sprites, while other nodes represent 
operations that manipulate or transform these sprites. The root node 306 is the desired output of 
the graph. 5 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply editing scanned documents disclosed by Bushey and Cunningham 
in combination with alternative graph as disclosed by Smith, and motivated to combine the 
teachings because it would show how converting progresses as revealed by Smith in [para. 
0047]. 

Allowable Subject Matter 
9. Claims 8, 12 and 15 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 
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Response to Arguments 

J_0. Applic^t^sjirg^^ — 

view of the new ground(s) of rejection. 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Responses 

12. Responses to this action should be mailed to: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231. 

Inquiries 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gregory F. Cunningham whose telephone number is (571) 272- 
7784. The examiner can normally be reached on Mon. - Thurs. 7:00 AM to 5:30 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor JMatthew Bellacan be reachedjHi (5_71)_272-7_778.-Jhe-fax phone-number for the — 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Gregory F. Cunningham 

Examiner 
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